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Abstract 

The massive collection of personal data by personalization systems has 
rendered the preservation of privacy of individuals more and more difficult. 
Most of the proposed approaches to preserve privacy in personalization 
systems usually address this issue uniformly across users, thus ignoring 
the fact that users have different privacy attitudes and expectations (even 
among their own personal data). In this paper, we propose to account for 
this non-uniformity of privacy expectations by introducing the concept of 
heterogeneous differential privacy. This notion captures both the variation 
of privacy expectations among users as well as across different pieces of 
information related to the same user. We also describe an explicit mecha¬ 
nism achieving heterogeneous differential privacy, which is a modification 
of the Laplacian mechanism by Dwork, McSherry, Nissim, and Smith. In 
a nutshell, this mechanism achieves heterogeneous differential privacy by 
manipulating the sensitivity of the function using a linear transformation 
on the input domain. Finally, we evaluate on real datasets the impact 
of the proposed mechanism with respect to a semantic clustering task. 
The results of our experiments demonstrate that heterogeneous differen¬ 
tial privacy can account for different privacy attitudes while sustaining a 
good level of utility as measured by the recall for the semantic clustering 
task. 


1 Introduction 

The amount of personal information about individuals exposed on the Internet is 
increasing by the second. While such data may be used for recommendation and 
personalization purposes [T1[2]|31I11[S] , this also raises serious privacy concerns. 
At first, leveraging personal information to enhance the user experience through 
personalization services might seems contradictory with the preservation of the 
privacy of users of such systems. However in recent years, several approaches 
have been proposed to rely on Privacy-Enhancing Technologies (PETs), whose 
aim is to preserve privacy while maintaining a good level of utility for the 
proposed personalization service [SlIZlIHlinilin]- One popular approach whose 
objective is to provide strong privacy guarantees despite auxiliary information 
that the adversary could have is the concept of differential privacy [anmiiia 
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Most of these approaches implicitly assume homogeneity by considering that 
users have uniform privacy requirements. However, in an environment composed 
of a myriad of communities, such as the Internet, it is highly plausible that users 
have heterogeneous privacy attitudes and expectations. For instance, consider 
a collaborative social platform in which each user is associated to a profile 
{e.g., a set of URLs that a user has tagged in a system such as Deliciou^. 
It is natural to expect that for a particular user some items in his profile are 
considered more sensitive by him than others, thus calling for a system that can 
deal with different privacy requirements across items. Similarly, Alice might 
be more conservative about her privacy than Bob, requiring different privacy 
requirements across users. 

This non-uniformity of privacy attitudes has been acknowledged by major 
social networking sites [ni[T8]. For instance in Facebook, a user can now set in¬ 
dividual privacy settings for each item in his profile. However in this particular 
example, privacy is mainly addressed by restricting, through an access-control 
mechanism, who is allowed to access and view a particular piece of informa¬ 
tion. Our approach can be considered to be orthogonal but complementary to 
access-control. More precisely, we consider a personalized service, such as rec¬ 
ommendation algorithm, and we enforce the privacy requirements of the user on 
its output. Heterogeneous privacy requirements might also arise with respect to 
pictures, depending on the location in which the picture was taken or the per¬ 
sons appearing on it [TS]. In the future, users are likely to expect item-grained 
privacy for other service^. 

Furthermore, as highlighted by Zwick and Dholakia in 1999 m and as 
evidenced by anthropological research, privacy attitudes are highly dependent 
on social and cultural norms. A similar point was raised in 2007 by Zhang and 
Zhao in a paper on privacy-preserving data mining [20] in which they mentioned 
that in practice it is unrealistic to assume homogeneous privacy requirements 
across a whole population. In particular, their thesis is that enforcing the same 
privacy level across all users and for all types of personal data could lead to 
an unnecessary degradation of the performance of such systems as measured in 
terms of accuracy. More specifically, enforcing the same privacy requirements 
upon all users (even those who do not require it) might degrade the performance 
in comparison to a system in which strict privacy requirements are only taken 
into account for those who ask for it. The same type of argument can also be 
made for different items of the same user. Hence, designing a system supporting 
heterogeneous privacy requirements could lead to a global improvement of the 
performance of this system as compared to a homogeneous version. Therefore, 
the main challenge is to be able to account for the variety of privacy requirements 
when leveraging personal data for recommendation and personalization. 

^http://del.icio.us/ 

^Note that systems supporting item-grained privacy can also provide user-grained privacy 
(he., for instance by setting the privacy level of all items in some user’s profile to the same 
value in the privacy setting of this user), and therefore the former can be considered as a 
generalization of the latter. However, this assumes that the privacy weights have a global 
meaning across the entire system, and are not defined only relative to a user. 
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In this paper, we address this challenge through the introduction of the 
concept of heterogeneous differential privacy, which considers that the privacy 
requirements are not homogeneous across users and items from the same user 
(thus providing item-grained privacy). This notion can be seen as an extension 
of the concept of differential privacy m introduced originally by Dwork in the 
context of databases. We also describe an explicit mechanism achieving het¬ 
erogeneous differential privacy, which we coin as the “stretching mechanism”. 
We derive a bound on the distortion introduced by our mechanism, which cor¬ 
responds to a distance between the expected output of the mechanism and the 
original value of the function to be computed. Finally, we conduct an exper¬ 
imental evaluation of our mechanism on a semantic clustering task using real 
datasets. The results obtained show that the proposed approach can still sus¬ 
tain a high utility level (as measured in terms of recall) while guaranteeing 
heterogeneous differential privacy. 

The outline of the paper is as follows. First, in Section [51 we describe the 
background of differential privacy as well as some preliminaries on matrices and 
sets necessary to understand our work. Afterwards in Section|31 we introduce the 
novel concept of heterogeneous differential privacy along with the description of 
an explicit mechanism achieving it. Then, we assess experimentally the impact 
of the proposed mechanism by evaluating it on a semantic clustering task in 
Section U) In Section [SI we present the related work on heterogeneous privacy 
mechanisms before concluding with a discussion on the actual limitations of the 
approach as well as possible extensions in Section |6l 

2 Background 

In this section, we briefly introduce the background on differential privacy (Sec¬ 
tion IQ) as well as some notions on matrices and sets that are necessary to 
understand the concept of heterogeneous differential privacy ((Section 12.2|1 . 

2.1 Differential Privacy 

We begin with providing some background of differential privacy, which was 
originally introduced by Dwork m in the context of statistical databases. The 
main guarantee provided by this approach is that if a differentially private mech¬ 
anism is applied on a database composed of the personal data of individuals, no 
output would become significantly more (or less) probable whether or not a par¬ 
ticipant removes this particular data from the dataset. In a nutshell, it means 
that for an adversary observing the output of the mechanism, the advantage 
gained from the presence (or absence) of a particular individual in the database 
is negligible. This statement is a statistical property about the behavior of the 
mechanism (z.e., function) and holds independently of the auxiliary knowledge 
that the adversary might have gathered. More specifically, even if the adver¬ 
sary knows the whole database but one individual row, a mechanism satisfying 
differential privacy still protects the privacy of this row. The parameter e is 
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public and may take different values depending on the application (for instance 
it could be 0.01, 0.1, 0.25 or even 2). While it is sometimes difficult to grasp the 
intuition about the significance of a particular value for e m, a smaller value 
of e implies a higher privacy level. 

Differential privacy was originally designed for ensuring privacy to individ¬ 
uals who have contributed with their personal data to the construction of a 
statistical database. In this setting, each individual is a row (i.e., coordinate) 
in this database («.e., vector). Differential privacy guarantees that almost no 
difference will be observed to the output of query performed on the database, 
whether or not the individual (a single row) has contributed to the database 
by submitting his data, and therefore this information is considered as being 
protected. 

When the database is the profile of a user, which is a vector of items (some¬ 
times called the micro-data setting), the whole vector (i.e., database) is owned 
by a single individual. This difference impacts the interpretation that can be 
done when speaking about protecting the privacy of this individual. In particu¬ 
lar, contrary to the first setting of statistical database, an individual does not 
have the choice to submit or not his data. Rather, if he chooses not to use his 
profile as input to the collaborative social system, then he will not benefit from 
the service. However in this new setting, the user is still left with the possibility 
of selecting a subset of items in his profile before participating. In this case, 
the main objective of differential privacy is to ensure that when a user adds or 
removes a single item from his profile, this has a small effect on the output of 
the computation. However, one caveat is that if the profile of the user contains 
nothing but items related to a particular sensitive topic (e.g., cancer), then in 
order to get at least a little bit of utility that information has to be leaked. This 
observation is in line with the impossibility result of Dwork and Naor stating 
that if a privacy-preserving mechanism provides any utility, then it has to cause 
a privacy breach whose magnitude is at least proportional to the min-entropy of 
the utility M- Thus, this limitation is true for any possible privacy-preserving 
mechanism and is not inherent to the micro-data setting {i.e., this limitation 
also holds for the database setting). 

The difference of a single row between two profiles can be defined formally 
through the concept of neighboring profiles. Each user is associated with a profile 
representing his personal data, which can be defined as a vector in R" (for some 
n fixed for all users across the system). This representation is generic enough 
to encompass a variety of possible user profiles. For instance, restricting the 
domain to {0,1}" can be used to represent a binary string (which is a universal 
representation) or a subset of items of a global domain of items. 

Definition 1 (Neighboring profile). Two profiles d,3~'^^ € K" are said to be 
neighbors if there exists an item i G {I,..., n} such that dk = d^^'^ for all items 
k ^ i. This neighborly relation is denoted by d 

An equivalent definition states that d and are neighbors if they are 
identical except for the f-th coordinate. For instance, the profiles (0,1,2) and 
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(0, 2, 2) are neighbored while the profiles (0,1, 2) and (0, 2, 3) are not. Differen¬ 
tial privacy can be defined formally in the following manner. 

Definition 2 (e-differential privacy [H]). A randomized function A4 : K" ^ R 
is said to be e-differentially private if for all neighboring profiles d ^ G K", 
and for all outputs t G M o/ this randomized function, the following statement 
holds: 

Pr[Ad(d) = t] ^ exp(e) Pr[Ad(d^*^) = t] , (1) 

in which exp refers to the exponential function. 

Differential privacy aims at reducing the contribution that any single coordi¬ 
nate of the profile can have on the output of a function. The maximal magnitude 
of such contribution is captured by the notion of (global) sensitivity. 

Definition 3 (Global sensitivity [H]). The global sensitivity S{f) of a function 
f is the maximum absolute difference obtained on the output over all neighboring 
profiles: 

S{f) = max |/(d) - /(c?*))| , (2) 

where means that d and are neighboring profiles ( cf. Definition\^. 

Dwork proposed a technique called the Laplacian mechanism |15] that achieves 
e-differential privacy by adding noise to the output of a function proportional 
to its global sensitivity. The noise is distributed according to the Laplace distri¬ 
bution (with PDF 2 ^ exp(—|a;|/(T), in which a = S(f)/e is a scale parameter). 

The novel mechanism that we propose in this paper (to be detailed later) 
achieves heterogeneous differential privacy by modifying the sensitivity of the 
function to be released (and therefore the function itself) before applying the 
standard Laplacian mechanism. 

2.2 Preliminaries 

Before delving into the details of our approach, we need to briefly introduce 
some preliminary notions on matrices and sets such as the concept of shrinkage 
matrix [22j . A shrinkage matrix is a linear transformation that maps a vector 
to another vector with less magnitude, possibly distorting it by changing its 
direction. 

Definition 4 (Shrinkage matrix). A matrix A is called a shrinkage matrix if 
and only if A = diag(ai,..., a„) such that each diagonal coefficient is in the 
range 0 ^ cti ^ 1. 

For example, the matrix 

!) 


/ 0.7 0 

0 0.3 

\ 0 0 


is a shrinkage matrix. 
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Definition 5 (Semi-balanced set). A set D C R” of column vectors is semi- 
balanced if and only if for all shrinkage matrices A = diag(ai,..., an), and for 
all X G D, we have Ax G D. 

For instance, the set 

{x = {xi,X 2 ) e I 0 < a;i,a ;2 < 1} (4) 

is a semi-balanced set that can be visualized as a square from (0,0) to (1,1) in 
the Euclidean plane. 

3 Heterogeneous Differential Privacy 

In this section, we introduce the novel concept of heterogeneous differential 
privacy (HDP). We start by giving the necessary definitions in Section 13.11 
before describing in Section 13.21 how to construct the Stretching Mechanism, 
which ensures heterogeneous differential privacy. More precisely, we first detail 
how to construct the privacy-preserving estimator in Section [3.2.1l Afterwards, 
we discuss why and how the privacy vector expressing the privacy expectations 
of a user should also be kept private in Section r3.2.3l Finally, an upper bound on 
the distortion induced by the Stretching Mechanism is provided in Section r3.2.4l 

3.1 Definitions 

We now define HDP-specific notions such as the concept of privacy vector^ which 
is a key notion in HDP. This vector contains the privacy requirements of each 
coordinate (be., item) in the input profile (be., vector) of a user, and is defined 
as follows. 

Definition 6 (Privacy vector). Given a profile d G D in which D is a semi- 
balanced set of column vectors composed of n coordinates, let v G [0,1]" be the 
privacy vector associated with the profile d. The owner of item di is responsible 
for choosing the privacy weight Vi associated to this item (by default Vi is set to 
be 1 if it was not explicitly specified by the owner). A privacy weight Vi of zero 
corresponds to absolute privacy while a value of 1 refers to standard privacy, 
which in our setting directly correspond to the classical definition of e-differential 
privacy. 

The mere presence of the privacy vector introduces potential privacy breaches, 
thus this vector should also be protected. Thus, we need to ensure that in addi¬ 
tion to the profile, the privacy vector v also remains private, such that each entry 
Vi of this vector should only be known by its owner. Otherwise, the knowledge 
of a privacy weight of a particular item might leak information about the profile 
itself. For instance, learning that some items have a high privacy weight may 
reveal that the user has high privacy expectations for and is therefore interested 
in this specific type of data. We define heterogeneous differential privacy in the 
following manner. 
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Definition 7 ((e,'!/)-differential privacy). A randomized function Ai : D ^ M. 
is said to be {e,v)-differentially private if for all items i, for all neighboring 
profiles d ~ , and for all possible outputs t S K o/ this function, the following 

statement holds: 

Pr[jV((d) = t] < exp(eui) = t], (5) 

in which exp refers to the exponential function. 

Since a privacy weight ^ 1, heterogeneous differential privacy implies the 
standard notion of e-differential privacy as shown by the following remark. 

Remark 1 (Equivalence of (e,u)-DP and e-DP.). Let e = sv and e = ev, such 
that V = maxi Vi (the maximum privacy weight) and v = min^ Vi (the minimum 
privacy weight). Then, we have: e-DP (e,v)-DP and {e,v)-DP 

e-DP. As a consequence, {e,l)-DP holds if and only if e-DP also holds, in 
which 1 = (1, • • ■ ,1). 

Finally, we rely on a variant of the notion of global sensitivity, implicitly 
introduced [231 Lemma 1], that we call modular global sensitivity. 

Definition 8 (Modular global sensitivity [53]). The modular global sensitivity 
Si{f) is the global sensitivity of f when d and are neighboring profiles that 
differ on exactly the item i. 

In a nutshell, the modular global sensitivity reflects the maximum difference 
that a particular item i can cause by varying its value (over its entire domain) 
while keeping all other items fixed. 

3.2 The Stretching Mechanism 

Thereafter, we describe a generic mechanism achieving heterogeneous differen¬ 
tial privacy that we coin as the Stretching Mechanism. We assume that the 
privacy preferences for each item are captured through a privacy vector v {cf. 
Definition [3|) . Given an arbitrary total function / : D —>■ R, in which D is a 
semi-balanced set of columns vectors of n coordinates, and whose global sensi¬ 
tivity S{f) is finite, we construct a randomized function f{d,v,e) estimating f 
while satisfying (e, u)-differential privacy. 

Before delving into the details of this method, we provide a little intuition 
on how and why it works. A lemma in [231 Lemma 1] asserts that the Lapla- 
cian mechanism M{d) = f{d) Lap((T) with mean 0 and standard deviation a 
provides 

Pr[A4(d) = f] < exp(ei) Pr[A4((?*^) = t] , (6) 

where = Si{f)/<j. In other words, differential privacy can be achieved by 
setting the perturbation induced by the Laplacian mechanism to be proportional 
to the modular global sensitivity [53] instead of the standard global sensitivity. 
Therefore, a natural approach for enforcing heterogeneous differential privacy 
is to manipulate the modular global sensitivity Si{f) by modifying the function 
/ itself. 
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3.2.1 Constructing the Estimator 

Let T : [0,1]" —)■ be a function taking as input a privacy vector v and 

returning as output a shrinkage matrix, with the property that T(l) = J, such 
that I is the identity matrix and 1 = (I,-- - ,1). Let also i? be a mapping 
sending a function /:£)—>■ R and a privacy vector v € [0, Ij" to the function 
i?(/, u) : Z? —> M with R{f, v)id) = f{T{v) ■ d). Recall that the Laplace distribu¬ 
tion centered at 0 with scale parameter a has the following probability density 
function 

= ^exp(-|a:|/cr) . (7) 

Finally, let TV be a Laplacian random variable with parameter cr = a{f,e) = 
S{f)/e, in which S{f) refers to the global sensitivity of the function / and 
£ the privacy parameter. The following statement proves that this Stretching 
Mechanism R satisfies heterogeneous differential privacy. 

Theorem 1 (Achieving HDP via stretching mecanism). Given a privacy vec¬ 
tor V, if the function T{v) satisfies Si{R{f,v)) ^ ViS{f) then the randomized 
function f{d,v,e) = R{f,v){d) -I- N satisfies {e,v)-differential privacy. 

Proof See Appendix. □ 

In a nutshell, T{v) is a shrinkage matrix, whose shrinking factor in each 
coordinate is computed independently of all other coordinates. More precisely, 
the shrinking factor for a particular item depends only on the privacy weight 
associated to this coordinate. The value used by the mechanism is the lowest 
amount of shrinkage {i.e., distortion) still achieving the target modular global 
sensitivity of that coordinate. In the following section we provide an explicit 
construction of T{v) for which we prove that by Lemma |T] the condition of 
Theorem |T] is satisfied, and therefore that / achieves (e, u)-differential privacy. 

3.2.2 Computing the Shrinkage Matrix 

The HDP mechanism /(d, 17, e) adds Laplacian noise to a modified function 

R{f,v){d) = f{T{v) ■ d). In this section, we specify how to construct T{v) 

such that / satisfies HDP. Thereafter, we use R to denote i?(/, v) for the sake of 
simplicity. Let T{v) = diag(rZ;) for some w £ [0,1]" to be computed from the pri¬ 
vacy vector V and S(R, w) be the sensitivity of i? = f{T{v) ■ d) = /(diag(ii;) • d) 
given w. Similarly, let Si (R, w) be the modular global sensitivity of R given w. 
We denote by {w-i^w'f} the vector resulting from replacing the item Wi in w to 
w[ {e.g., (1-i, Wi) = (I,..., Wi ,...,!)). Each Wi can be computed from Vi by 
solving the following optimization problem: 

max Wi , , , 

subject to: Si{R, {l-i,Wi)) ^ ViS{f) . 

Note that a solution satisfying this constraint always exists and is reached 
by setting Wi to 0. The wfs are never released after they have been computed 
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locally by the rightful owner, and the modular global sensitivity Si{R) is only 
used in the proof and is not revealed to the participants, in the same manner 
as the noise generated. The participants only have the knowledge of the global 
sensitivity S{f). Thus, the only way in which the profile d could leak is through 
its side effects to the output, which we prove to achieve e-DP in Theorem [51 

Lemma 1. IfT(v) = diag(r(;) such that for all i: 

Si{R, (l_i, Wi)) ^ ViS{f) (9) 

(the constraint of then R satisfies: 

Si{R,w) !fViS{f) (10) 

for all i. 

Proof. See Appendix. □ 

3.2.3 Hiding the Privacy Vector 

By themselves, the privacy weights could lead to a privacy breach if there are 
release publicly nm. For instance, learning that the user has set a high weight 
on a particular item might be indicated that the user possesses this item on his 
profile and that he has a high privacy expectation about it. Thus, the impact 
of the privacy weights on the observable output of the mechanism should be 
characterized. The following theorem states that when the profile d is fixed, the 
randomized function / satisfies ^-differential privacy over neighboring privacy 
vectors v ~ Thus, the privacy vector can also be considered to be hidden 
and protected by the guarantees of differential privacy. 

Theorem 2 (Protecting the privacy vector with e-DP). The randomized func¬ 
tion f provides s-differential privacy for each individual privacy weight of v. 
This means that for all neighboring privacy vectors v ^ for all outputs 
t S K and profiles d, the following statement holds: 

Pr[fid,v,e) = t] < exp(£)Pr[/(d,F(®),e) = t] . (11) 

Proof. See Appendix. □ 

3.2.4 Estimating the Distortion Induced by HDP 

Intuitively, if w is the minimum of w (the diagonal of T(u)), and d is the input 
profile, then the distortion introduced by stretching the function as measured 
in terms of absolute additive error is bounded by 1 — w times the norm of d 
multiplied by the norm of the gradient of the semi-stretched function at d. 

More formally, let / be a continuous and differentiable function on a semi- 
balanced set D, and let {v, d) € [0,1]" x D be respectively, the privacy vector and 
the profile considered. The following theorem provides a bound on the distortion 
introduced on the output by modifying the global sensitivity of the function / 
as done by the HDP (z.e., stretching) mechanism described previously. 
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Theorem 3 (Bound on the distortion induced by the Stretching Mechanism). 
Let f : D ^ W be a function from a semi-balanced set D to the reals, and let 
V G [0,1]" be a privacy vector and T : [0,1]" ^ R"^" be a function taking a 
privacy vector to a shrinkage matrix. Finally, let R he a mapping sending a 
function f and a privacy vector v to the function R{f, v) : D ^ M. such that 
R{f,v){d) = flT[v) -d) for all vectors d. The distortion (\.e., distance) between 
f and R{f, v) is bounded by: 

\f{d)-R{f,v){d)\^ m&Ml-w)\\Vf{B-m\d\\ , (12) 

where B = cl {1 — c)T{v), w = min^ Wi is the minimum of w (the diagonal of 
T(v)), and 'V f is the gradient of the function f. 

Proof. See Appendix. □ 

This bound is particularly useful in situations in which the norm of the 
gradient of the function / is bounded from above by a constant. However, even 
if the norm of the gradient is not bounded by a constant, the bound can still 
be useful. For instance, in the case of the scalar product function, the bound 
on the distortion will be (1 — :w)|ldj|^, due to the fact that the gradient of the 
scalar product function is equal to ||i3-(i|| < ||ci|| (since H is a shrinkage matrix). 
One restriction on the application of this bound is that the function / to be 
protected should have a finite global sensitivity, and therefore the scalar product 
function mentioned has to restrict its domain to be finite, thus preventing the 
distortion bound from being infinite. 


4 HDP in Practice 

To assess the practicality of our approach, we have applied the HDP mechanism 
on a collaborative social system [I] , and evaluated its impact on a related seman¬ 
tic clustering task. In this collaborative social system, each user {i.e. node) is 
associated with a profile. A profile is the set of items the user has liked or tagged 
{e.g., the set of URLs in his Delicious account). The objective of the semantic 
clustering task is to assign each node with the A:-closest neighbors according to 
a given similarity metric. In this paper, we use the classical cosine similarity 
(introduced later) to quantify the similarity between two profiles. The task is 
carried out using a fully distributed protocol, therefore the nodes compute lo¬ 
cally {i.e., without relying on a central authority) their similarity with other 
profiles. 

4.1 Applying HDP to Semantic Clustering 

In the context of distributed semantic clustering, we are interested in providing 
heterogeneous differential privacy guarantees to the profiles of nodes {i.e., users). 
More precisely, we consider the scenario in which a particular user can assign 
a privacy weight, between 0 and 1, to each item of his profile. The value 0 
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corresponds to the strongest privacy guarantee in the sense that the presence 
(or absence) of this item will not affect the outcome (the clustering) at all, while 
the value 1 is the lowest level of privacy possible in our framework (however 
it still provides the standard guarantees of e-differential privacy). Thus, the 
privacy weights of a user directly reflect his privacy attitudes with respect to 
particular items of his profile, and as a side effect determines the influence of 
this item in the clustering process. In particular, an item with a higher weight 
will contribute more to the clustering process, while a item with a lower weight 
will influence less the resulting clustering. 

The cosine similarity between two profiles X and Y is defined as 

cos_sim(X, F) = —, (13) 

\/l^l X |r| 


such that |X n F| is the number of items in common between X and Y, and 
|X| and |F| correspond to the number of items of X and Y, respectively. 

The indicator function of a profile, when it is represented as a binary vector, 
for the item z is 1 if the item is present in the profile and 0 otherwise. More 
formally, the z*^ coordinate Xi(x) of the indicator function x{^) of ffio profile x 
is denoted by: 


f 1 if z G X 

1 0 otherwise 


(14) 


Using the notation of the indicator function, the cosine similarity could be 
defined as 


X{X) ■ x{Y) 
I|X(^)I|2||X(T)||2 ’ 


(15) 


in which the operation denotes the scalar product. In the following, we apply 
HDP to the scalar product function and use this modified version to compute 
the cosine similarity on profiles represented as binary vectors. 

Given two prohles X and Y and their corresponding indicator functions 
X = x(X) and y = x(F), let SP{x,y) = refers to the scalar product 

between the two profiles. The privacy vector v is composed of two parts, one 
for the prohle x and the other for the profile y: (iX^iP). Consider the matrix 
T{v) = diag(z;) and let R{SP,v) = SP(T(u®) • x,T{iP) ■ y) be the Stretching 
Mechanism, in which T is the stretch speciher. This mechanism R satisfies the 
premise of Theorem [1] and therefore the choice of T{v) = diag(u) also ensures 
HDP, as proven in the following lemma. 


Lemma 2. Consider a matrix T{v) = diag(z/) and a mechanism i?(SP,z/) = 
SP(T(u®) • x,T{iP) ■ y), such that x and y correspond to profiles and and 
to their associated privacy vectors. In this situation, the following statement is 
always true: Si{R{SP,v)) ^ z;iiS'(SP) for all i. 


Proof. See Appendix. 


□ 


The previous lemma proves that the proposed modified version of scalar 
product is differentially private, while the next lemma simply states that if 
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we rely on this differentially private version of scalar product to compute the 
cosine similarity (or any similar metric), the outcome of this computation will 
still be differentially private. A standard {i.e., non-heterogeneous) version of 
the following post-processing lemma can be found in the literature [25) . which 
we have generalized to heterogeneous differential privacy. 

Lemma 3 (Effect of post-processing on HDP). If a randomized function f satis¬ 
fies {e,v)-differential privacy, then for any randomized function g : Range(/) — >■ 
K. independent of the input, the composed function g o f satisfies also {s,v)- 
differential privacy. The randomness of the function g is assumed to be inde¬ 
pendent of the randomness of f in order for this property to hold. 

Proof. See Appendix. □ 

4.2 Experimental Evaluation 

For the experiments, we assume that in real life, nodes will assign different 
privacy weights to the items in their profiles. In order to simulate this, we 
generated privacy weights uniformly at random from a set of n equally-spaced 
values in a fixed range [u, u]. More formally, each item is associated with a 
privacy weight sampled uniformly at random from the set {u,u-bd, ... ,u — S,u}, 
6 = {u — u)/{n — 1), for 0 ^ ^ 1. For instance, if u = 0.5, u = 1 and 

n = 3, then the weights assigned to items will be uniformly chosen from the set 
{0.5,0.75,1}. 

We run our experiments on three datasets coming respectively from Deli¬ 
cious, Digg and a survey conducted within our lab. About 120 users partici¬ 
pated in the survey and submitted their feedback (in forms of like/dislike) on 
approximately 200 news. Therefore, in the survey dataset a user’s profile con¬ 
sists of the news he has liked, while for the Digg dataset a profile consists of 
the items that a user has forwarded to others users. Finally, in the Delicious 
dataset, the profile of the user consists of the items he has tagged. 

• Delicious dataset. Delicious (delicious.com) is a collaborative platform 
for keeping bookmarks in which users can tag the URLs of websites they 
liked. The Delicious dataset consists in the profiles of approximately 500 
users, a profile being a set of URLs that the user has tagged. The total 
number of URLs in the collective set of users’ profiles is over 50,000 URLs. 
In such a setting, the problem of similarity computation arises naturally, 
when providing personalized services such as the recommendation of URLs 
drawn from the ones tagged in Delicious. For the sake of simplicity, in the 
experiments conducted, each URL was assigned a unique identifier in the 
range of (1,..., 50000}, in order to handle identifiers as integers instead 
of URL strings. The average size of a profile is 135 URLs. 

• Digg dataset. The dataset consists of 500 users of Digg (digg.com), a 
social news website. The profile of these users is composed of the news that 
they have shared over a period of 3 weeks in 2010. All the users considered 
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have shared more than 7 items per week and the dataset contains 1250 
items, each of which has been shared by at least 10 users. The average 
size of a profile is 317 items. 

• Survey dataset. Around 200 randomly chosen news on various topics have 
been shown to 120 colleagues and relatives, who have then submitted 
their opinion in terms of like/dislike for each news. The average size of 
the profile is 68. Indeed, while each user has answered to all the 200 pieces 
of news, he has only liked 68 of those pieces of news on average. 

The distributed clustering algorithm is gossip-based and works in an iterative 
manner [1]. In order to assess the quality (he., utility) of a particular clustering, 
we rely on the recall metric. The recall can be defined as the ratio between 
the number of search items a node could find in the profiles of his k closest 
neighbors (as induced by the clustering) over all possible items of his profile. 
We consider this metric for our experiments but other standard metrics used 
in recommendation systems could work as well. In the experiments conducted, 
the profile of each user is split at random into a training set composed of 90% 
of the profile while the remaining 10% is used for testing. After 20 rounds of 
exchanging gossip messages during the clustering protocol, each user searches 
for those 10% of items in the profiles of the k closest neighbors provided by the 
clustering protocol. The recall is then equal to the ratio of items found in the 
profiles of the neighbors over all the possible items contained in the testing set. 
The average recall of all users is then reported as the outcome of the experiment. 

In Figure [U we have plotted the three cases for which the interval {u^u) is 
set to be (0,1), (0.5,1), and (0.9,1). The x-axis represents u, while the y-axis is 
the recall averaged over all slices (from n = 1 to n = 10) for the experiment in 
the range [x, 1]. Afterwards in Figure^ we have fixed the range u G {0, 0.5, 0.9} 
and 7t = 1 and plot the average recall over all users over all runs versus n, the 
number of slices (ranging from 1 to 10). In both figures, the error bars represent 
the variance. 

From Figured] (Delicious), we can observe that there is not much difference 
in terms of utility between the situations in which u = 0.5 and u = 0.9, as both 
situations are close to the utility obtained with the baseline algorithm. Indeed, 
the largest difference is obtained when u is set to 0, in which case the utility 
gets closer from the utility obtained through a random clustering. Furthermore, 
Figure [2](Delicious) demonstrates that varying the number of slices has almost 
no effect on the utility achieved by u G {0.5,0.9}, but has significant impact on 
the situation in which u = 0, for which the utility decreases as the number of 
slices increases. One possible interpretation is that as n (the number of slices) 
increases, there are more and more items whose privacy weight differs from 1. 
In a nutshell, this seems to indicate that items with high privacy weights (above 
0.5) have an important impact on the utility. Combining this observation with 
the fact that when u ^ 0.5, the utility was not affected show that items with low 
privacy weights (less than 0.5) can harm the utility in a non-negligible manner. 
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Figure 1: The value reported is the average recall obtained when all peers 
have the same distribution over privacy weights for all items, averaged over the 
number of slices. Baseline refers to the recall obtained when the system run 
with no privacy guarantees using the plain version of the clustering algorithm, 
while Random refers to a random clustering process in which peers choose their 
neighbors totally at random. 
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Figure 2: The value reported is the average recall obtained when all peers have 
the same distribution over privacy weights for all items, plotted against the 
number of slices. 
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While this may seem strange at first glance, it actually solves an apparent 
contradiction when taken from a different point of view. Consider for instance 
that a privacy weight is changed by 0.1 {e.g., from 0.9 to 0.8 or from 0.1 to 0.2). 
The amount of impact on utility resulting from this change depends not only 
on the value of the difference, but also on the original value being changed. On 
one hand, the utility gain resulting from modifying the privacy weight from 0.1 
to 0.2 is less than the utility loss if the privacy weight were modified from 0.9 
to 0.8. On the other hand, changing u from 0.5 to 0 can cause a significant 
damage to utility because the average privacy weight drops from 0.75 to 0.5. 

4.3 Varying Privacy Attitudes Among Users 

The results of the previous section were obtained for the setting in which all 
nodes draw their privacy weights from the same distribution (he., all users have 
the same privacy attitude). However, according to a recent survey [2^, users 
of information systems can be classified in at least three very different groups 
called the Westin categories [27] . These three groups are: Privacy Funda¬ 
mentalists, Privacy Pragmatists and Privacy Unconcerned. The first 
group is composed of the users concerned about their privacy, while on the 
contrary the third group is composed of the ones that are the least concerned 
(according to a particular definition of concern detailed in the cited poll) and 
finally the second group is anything in between. For the following experiments, 
we have adopted the spirit of this classification and consider the three groups 
of users defined thereafter. 

Each group is equipped with a different distribution from which they pick 
their privacy weights as follows. 

1. The Unconcerned group corresponds to users that do not really care 
about their privacy and thus all their items have a privacy weight of 1. 

2. The Pragmatists group represent users that care a little bit about their 
privacy, such that all their items have a privacy weight chosen uniformly 
at random among {0.5,0.75,1}. 

3. The Fundamentalists group embodies users that really care a lot about 
their privacy and whose items have a privacy weight chosen uniformly at 
random among {0, 0.5,1}. 

The main issue we want to investigate is how the presence of a relatively conser¬ 
vative group (i.e., having relatively high privacy attitudes) affect the utility of 
other groups. More specifically, we want to measure whether or not the presence 
of a group of nodes with high privacy attitudes indirectly punish (i.e., reduce 
the utility) of other more open groups. 

During the experimentations, we have tried different proportions of these 
groups for a total number of users of 500. Each value plotted in Figure [S] 
has been averaged over 10 runs but the partition in groups is fixed for a given 
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set of runs. All experiments are averaged on e £ {0.1,0.5,1, 2,3}. Accord¬ 
ing to a 2004 poll [50], the percentage of each of the privacy groups Fun¬ 
damentalists, Pragmatists and Unconcerned are respectively, 34%,43% 
and 23%. Nonetheless, we also experiment a combination of several other dis¬ 
tributions in order to investigate other possible settings. In particular, we 
have also tried the following percentages for each group: the proportion of 
the Unconcerned group and Pragmatists group vary in the following range 
(10%, 20%, 60%, 70%}, while the Fundamentalists group is assigned to the 
remaining percentage (ie., there is only two degrees of freedom). If Uncon¬ 
cerned group -I- Pragmatists group > 100%, then this combination is dis¬ 
carded. In Figure [S] the a:-axis represents the percentage of the Fundamen¬ 
talists group, while the y-axis corresponds to the recall. Each of the three 
lines correspond to the recall of one of the three groups (Fundamentalists, 
Pragmatists, and Unconcerned). For each of the four plots, the proportion 
of the Pragmatists group is denoted in the plot by the expression Pragmatists 
= some value. The proportion of the remaining group (Unconcerned) can be 
directly inferred by subtracting the proportions of the two other groups from 
100%. 

From the results obtained, we can conclude that (1) Pragmatists and 
Unconcerned always have better recall than Fundamentalists and (2) Un¬ 
concerned often have a better recall than Pragmatists, though not always. 
This seems to indicate that the group caring more about privacy usually is pun¬ 
ished more (i.e., its utility is low) than groups that are more liberal with respect 
to privacy expectations. This not really surprising as a low privacy weight will 
result in users from the Fundamentalists group segregating themselves from 
other users in the clustering to the point that they will not necessarily have 
meaningful neighbors in their view. Finally, to the question whether (or not) 
more liberal groups will be punished by conservative groups, the answer seems 
to be negative. Indeed it can be seen from the results of the experiments, that 
conservative groups are punished more than liberal groups. For instance, the 
utility of liberal groups only decreases from 0.22 to 0.19 as the percentage of 
conservative groups increases from 20% to 80%. 


5 Related Work 

The majority of previous works on heterogeneous privacy has focused only on 
user-grained privacy [281129] , in which each user may define his own privacy level 
(instead of having the same privacy guarantee for all users across the system). 
As opposed to item-grained privacy, which allows each item of an individual 
user to have a different privacy weight, user-grained privacy restricts all the 
items of the same user to the same privacy weight. For instance. Das, Bhaduri, 
and Kargupta [28] have proposed a secure protocol for aggregating sums in a 
P2P network. In this setting, each node has an input vector, which could be for 
instance a profile. In a nutshell in this protocol, each node picks at random a 
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Figure 3: Results obtained for the Delicious, Digg and survey datasets. The 
heterogeneous differential privacy has been computed for 3 groups with different 
privacy attitudes. For a particular figure and a particular x tick, the percentage 
of Unconcerned group is fully determined as (1— Pragmatists — x). 
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few other nodes of the system with whom it computes some local functior(^ in a 
private manner (the local function begins with a sum as well). The more peers a 
specific node chooses to participate to the computation, the higher the privacy 
will be obtained by this node according to the considered definition of privacy. 
More precisely in their setting, privacy is mainly quantified by the probability of 
collusion of the peers chosen by a particular node when the aggregation protocol 
is run. This probability can be made smaller by choosing a larger set of peers, the 
main intuition being that for a particular node running the aggregation protocol 
with a larger group diminishes the probability that all these nodes will collude 
against him. Thus, the best privacy guarantees could be obtained by running 
the protocol with the entire set of peers but this would be too costly in practice. 
The main objective is this protocol is to be adaptive by providing a trade-off 
between the privacy level chosen by a user and the resulting cost in terms of 
computation and communication. In particular, each user has the possibility to 
choose heterogeneously the peers with whom he wants to run the aggregation 
protocol by taking into account his own privacy preferences. However, this work 
does not seem to be easily extendable to integrate item-grained privacy. 

Another work due to Kumar, Gopal, and Garfinkel [29] is a form of gener¬ 
alization of fc-anonymity |30| . The standard definition of /c-anonymity requires 
that in the sanitized database that is released, the profile of a particular individ¬ 
ual should be indistinguishable from at least k — 1 other individuals (thus here 
k can be considered as being the privacy parameter). The proposed generaliza¬ 
tion [29] essentially enables each user to require a different value for k for each 
attribute in his profile. For example, a user may require that his data should 
be included in the published database only if there are at least 4 other users 
sharing his ZIP code and at least 8 other users whose age difference with him 
is at most 3 years. The possibility of setting the range of a particular attribute 
could be regarded as item-grained heterogeneous privacy in the sense that an at¬ 
tribute whose privacy range is large is less likely to be useful for de-anonymizing 
the user than less private attribute. To summarize, the main objective of this 
approach is to protect the privacy of a user by anonymizing it {e.g., to prevent 
de-anonymization and linking attacks), while in our work the main objective 
is to prevent the possibility of inferring the presence or absence of a particular 
item in the profile. 

A line of research on auctions for privacy has provided almost the same 
definition for the heterogeneous differential privacy as ours [23l[24]. The main 
difference with our contribution is that these previous works do not provide a 
mechanism to realize heterogeneous difference privacy, but instead only use the 
definition to achieve the post-release privacy guarantees. In the model studied, 
the participants are composed of a data analyst and a group of users. Each 
user has as input a private bit and the data analyst wants to estimate in a 
differentially-private manner a global function of the private bits of all users, 
such as the sum or the weighted sum. The data analyst is willing to pay each 

®The function is local in the sense that it depends only on the inputs of the node and the 
peers it has chosen. 
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user for the loss of privacy he incurred by participating to this process. More 
precisely, each user i has a privacy valuation Vi(ei) : ]R+ K.+ indicating the 

amount of his loss given the privacy guarantee he gets. The user has no control 
over £i {i.e., the privacy guarantee he ends up with), which is decided solely by 
the auction mechanism. As such the valuation function Vi merely affects the 
payment of the user, as his payment is decided indirectly by the mechanism 
given the valuation function and is not decided directly by him. Therefore, our 
work is incomparable to theirs, because the privacy parameter £i acts mainly as 
an indication about the level of privacy reached, while in our setting the privacy 
parameter represents the user’s requirement about the privacy of a particular 
item of his profile. Moreover, in [53] it is stated that users finally end up having 
completely homogeneous privacy guarantees. More precisely, each user ends up 
having £-differential privacy, with some e being the same for all users. In contrast 
in [23] ■ users effectively have heterogeneous privacy guarantees. However, these 
guarantees are determined by the public weights of the auctioneer, which the 
auctioneer chooses so as to compute the weighted average of the users’ inputs 
and independently of the privacy valuations of the users. 

Finally, Nissim, Raskhodnikova and Smith m have investigated how the 
amount of noise necessary to achieve differential privacy can be tailored by 
taking into account to the particular inputs {i.e., profiles) of participants, in 
addition to the sensitivity of the function considered. The main objective of 
this approach is to reduce the amount of noise that needs to be added to in¬ 
puts that are not locally sensitive {i.e., for which the output does not change 
much if only one item is changed). However, they also show that the amount 
of noise added may itself reveal information about the inputs. Hence, they de¬ 
fined a differentially private version formalizing the notion of local sensitivity 
called smooth sensitivity, guaranteeing that the amount of noise added is itself 
£-differentially private. Similarly, we have ensured that for our notion of hetero¬ 
geneous differential privacy, the amount of noise added is not impacted by the 
specific profile considered or by the privacy requirements formulated by a user. 
Rather, we have modified the function under consideration and its sensitivity, 
which also impacts the distortion induced of the output (c/.. Section 13.2.41) . We 
have also proven that the privacy requirements of a user expressed in the form 
of private weights remain private are they are also covered by £-difFerentially 
privacy guarantees. Thus, it is difficult for an adversary observing the output of 
an heterogeneous differentially private mechanism to guess the privacy weight 
that a user has put on a particular item of his profile. 


6 Conclusion 

In this work, we have introduced the novel concept of heterogeneous differential 
privacy that can accommodate for different privacy expectations not only per 
user but also per item as opposed to previous models that implicitly assume 
uniform privacy requirements. We have also described a generic mechanism 
achieving HDP called the Stretching Mechanism, which protects at the same 
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time the items of the profile of user and the privacy vector representing his 
privacy expectations across items of the profile. We applied this mechanism for 
the computation of the cosine similarity and evaluate its impact on a distributed 
semantic clustering task by using the recall as a measure of utility. Moreover, 
we have conducted an experimental evaluation of the impact of having different 
groups of users with different privacy requirements. 

Although the Stretching Mechanism can be applied to a wealth of functions, 
it is nonetheless not directly applicable to some natural functions, such as the 
norm and min. Indeed, when computing the £o norm (i.e., the number of non¬ 
zero coordinates in a given vector), each coordinate contributes either zero or 
one regardless of its value. Since the Stretching Mechanism modifies this value, 
this mechanism would always output the true exact value as long as no privacy 
weight has been set to exactly zero. For the case of min, due to the fact that 
the Stretching Mechanism shrinks each coordinate by a factor corresponding to 
its privacy weight, the resulting output may not have anymore a relation to the 
intended semantics of the function min. 

Another challenge is to enable users to estimate the amount of distortion 
in the output that they received out of an heterogenous differentially private 
mechanism. For instance, for functions such as the sum, recipients will not be 
able to estimate the correct value without being given the distortion. Although 
the distortion has an upper bound given by Theorem [31 the information needed 
to compute the upper bound is private. Therefore, releasing the distortion (or 
even its upper bound) would constitute a violation of privacy. We believe this 
issue could be solved partially by releasing an upper bound using the traditional 
Laplacian mechanism at an additional cost of an £ amount of privacy. Another 
important future work includes the characterization of functions that have a 
low and high distortion. Indeed, functions having a high distortion are not 
really suitable for our HDP mechanism. We also leave as open the question of 
designing a different mechanism than the Stretching Mechanism achieving HDP 
with a lower distortion. 
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A Proofs 

A.l Proof of Lemma [H 

Proposition 1 (Monotonicity of subdomain optimization). Let 0 and 0' be 
the result of two maximization problems pi and p 2 of the function g in which 
the maximization is over domains J and J', respectively. Then, if J C J', 
this implies that 0^0'. The opposite statement also holds for minimization 
problems. 

Proof. Since 0' is the optimal result of p 2 over J', this means that by definition: 

gi^') > gU) > for all j in J'. (16) 

Moreover, since any result 0 for pi will always be in J, and therefore in J', 
then g{0') ^ g{0) by (flBl) . (The proof that the opposite statement holds for 
minimization problems follows from the same arguments and thus we choose to 
omit it.) □ 
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Lemma 4 (Shrinkage matrices composition). If A and B are two shrinkage 
matrices and D a semi-balanced set, then ABD C BD C D. 

Proof. By definition of semi-balanced set, we have BD C D. Then it remains 
to prove that ABD C BD (or equivalently, that BD is a semi-balanced set). 
We observe that a vector w belongs to ABD if and only if ic = ABb for some 
b G D. Because shrinking matrices commute, w = BAb. Let a = Ab. By 
definition of semi-balanced set, a G D. Therefore, w = Bet for a G D, which 
means w belongs to BD by definition oi BD. □ 

Lemma 5 (Monotonicity of the global sensitivity). If w' ^ w then S{R,w') ^ 
S{R, w). 

Proof. Let c be such that 


C (17) 

10 otherwise 

and let C = diag(c) is a shrinkage matrix. Then w' = Cw. Let T' = diag(w') 
and T = diag(?a) be two other shrinkage matrices. Notice that T' = CT. By 
Lemma [4] and since D is semi-balanced: 

T'D = CTD CTDCD. (18) 

The result follows from Proposition [T] because S{R,w) is over the domain 
TD while S{R, w') is a maximization problem over the domain T'D G_TD. □ 

Corollary 1 (Monotonicity of the modular global sensitivity). If w' ^ w then 
Si{R,w') ^ Si{R,w) for all i. 

Proof. By Lemma[5l we have that S{R, w') ^ S{R, w). Let i* = argmaxj Si{R, w) 
and therefore S{R,w) = Si*(R,w). In order to get a contradiction, we assume 


that Si*{R,w') > Si*{R,w), thus we have 

S{R, w') = maxSi{R, w') > Si* {R, w) = S{R, w), (19) 

i 

which is a contradiction. □ 

Proof o/LemmoQJ Since w ^ {l-i,Wi) for all i, then: 

Si{R,'w) < S^{R,{l-i,Wi)) for all i, (20) 

< ViS{f) for all i, (21) 

where the first inequality follows by Corollary[T]and the second inequality follows 
from the premise of the lemma, thus concluding the proof. □ 


24 


A.2 Proof of Theorem 

Proposition 2 (Semi-balanced sets are closed under shrinkage.). If D is a 
semi-balanced set and A is a shrinkage matrix, then AD is also a semi-balanced 
set. 


Proof. AD is semi-balanced if and only if for any shrinkage matrix B the fol¬ 
lowing is true: B{AD) C AD. Indeed, since B is a. shrinkage matrix and D is a 
semi-balanced set, then BD C D. By multiplying both sides by a shrinkage ma¬ 
trix A, we obtain ABD C AD, which by the fact that A and B are commutative 
implies that BAD C AD. □ 

Proposition 3 (T(h') and are neighbors). T{v).d and ■ d are 

neighbors on item i, since Wi is a function of Vi only. 

Proof of Theorem[B Let cT* =T{v) ■ d and ■ d. By Proposition [3] 

{cf. Section [5.2.2L d* and are neighboring profiles. Moreover due to Propo- 
sition[21 they still belong to D, thus \f{d*) — f{d!f'^)\ ^ Si{f) ^ S{f). Therefore, 
we have: 


PAfid,v,e) = t] 

Pr[/(d,d(d,e) = t] 


^ h{t-R{f,v){d)) 

h{t-R{f,m){d)) 

^e\R{f,v)[d)-R{f,v^^){d)\, 
^ Sif) > 


= exp( 


S{f) 

^ exp(^^^) = exp(e). 


where h{-) is defined in Q, thus proving the result. 


( 22 ) 

(23) 

(24) 

(25) 

□ 


A.3 Proof of Theorem [T] 

Proof of Theorem [H For all two neighboring profiles d, d^*^, and for all outputs 
t e M of the function / we have 


Pr[/(d, V, e) = t] 
Pr[/(d(d,F,e) = t] 


^ hit-R{f,v){d)) 
h{t - i?(/,{;)(dW)) 

^ ^^^^^^^ e\R{f,v)id) - Rif,v)i^n ^ 


Sif) 


^ exp( 


sSiiRif,v)) , 

Sif) ' 


^ = exp(eui), 


m 


(26) 

(27) 

(28) 
(29) 


where d(-) is defined in ([7]), thus proving the result. 


□ 
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A.4 Proof of Theorem [3] 

Proof of Theorem\^ Let y = d and x = T(v) ■ d, then by the mean value 
theorem [311 Theorem 14.4, p. 301], there exists a constant 0 ^ c ^ 1 (depending 
on d,T(v), and /) such that: 

f{y) - fix) = V/((l - c)x + cy) ■ {y- x), (30) 


where • denotes the scalar product. Therefore, by the Cauchy-Schwarz inequal¬ 
ity, we have: 

\fiy)-fix)\^ ||V/((1 - c)f-bcy)||||zr-x||. (31) 

Finally, by the fact that 


||y - f|| = \\d- T(F) • d\\ = 11(7 - Tiv)) • d|| (32) 


= - Wi)d^)^ 

^ (l-minw,) jY^i = (1 - 


w 


(33) 

(34) 


where w is the minimum of w (the diagonal of T(h')),and that 

(1 — c)x + cy = (1 — c)T(if) ■ d + cd (35) 

= icI+{l-c)T{v))-d, (36) 

the theorem follows directly. □ 


A.5 Proof of Lemma [2] 

Proof of Lemma\^ Each profile being represented as a binary vector, the global 
sensitivity of the scalar product is one {i.e., <S'(SP) = 1). Thereafter, for the sake 
of simplicity, let R denotes R{SP,v). As T{v) is a diagonal matrix, it is strictly 
identical to its transpose T{v)^. We can assume without loss of generality that 
= (f, for item j = i — dim(x), and therefore that: 


Si (R) = max \T{'if)x ■ T{dP)y - T{if)x ■ \ (37) 

dr^dM 

= max\{x^T(iF)T{iF)) ■ {y-if'^'>)\, (38) 

however the vector y — has all its coordinates set to 0 except for the co¬ 
ordinate. Therefore, the maximum is reached when = 1, x = 1 = (1, • • • , 1), 
and is such that: 

hfuf < Xi = Xi X 1 = XiS'(SP), (39) 

which concludes the proof. □ 
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A.6 Proof of Lemma [3] 

Proof of Lemma O The theorem is equivalent to prove that for any two neigh¬ 


boring profiles d ^ the following holds: 

Prb o fid) =t]^ exp(e'!;0 Pr[g o = t]. (40) 

To prove this, consider any two neighboring profiles d^ 

Pr [5 o f{d) =f\= J Pr[/(d) = s] ■ Pr[ 5 (s) = f\ (41) 

seRange(/) 

^ J exp{ev,) Pr[/(f?*)) = s] ■ Pr[ 5 f(s) = t] (42) 

seRange(/) 

= exp(ez;*) Pr[g o /(f?*^) = t], (43) 

thus concluding the proof. □ 
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